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Preface 


This is the second volume of a two volume NBS/NASA report on an Overview 
of Artificial Intelligence and Robotics . This report is intended to be read 
by the interested technically-oriented layman, such as engineering managers, 
government funding officers, and others who desire an overall perspective of 
the field but are unable to obtain it frem the highly technical and unintegrated 
literature in the field, or from the more flamboyant but non-tech-.-ical feature 
articles -tn the popular press. 

The report incorporates a summary of the basic concepts utilized in each 
of the many technical areas, a review of the state-of-the art, an indication 
of the organizations involved together with their funding sources and principal 
activities, research requirements, and a 5-10 year forecast of the emerging 
technology. 

As robotics is a very rapidly changing field - with new companies, products 
and researchers entering the field each month - a report like this can only be 
regarded as an imperfect -snapshot in time, and would benefit from periodic up- 
dating. Thus, quantitative information shown in this report may perhaps best 
be regarded as indicative rather than authoritative. Despite such shortcomings, 
this report should be helpful in providing a base, for organizations and individuals 
in and out of the government, for planning future robotic research and develop- 
ment, applications and utilizations. 
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ROBOTICS 


I. Introduction 

Robots in use today are primarily machines with manipulators that can be 
easily programmed to do a variety of manual tasks automatically. The robot con- 
sists of one or more manipulators (arms), end effectors (hands), a controller, 
power supply, and - increasingly - sensors to provide information about the en- 
vironment and feedback of performance in task accomplishment. The irptivation 
for most current robotic development is industrial: to increase productivity, 

reduce costs, overcome skilled labor shortages, provide flexibility in batch 
operations, improve product quality and to free humans from boring, repetitive 
tasks or operations in hostile environments. 

Table 1 gives the Japanese standard classifications for industrial robots 
(IR’s) by input information and teaching method, and relates this to the U.S. 
classification by control method - non-servo or servo. 

As indicated in Table 2, the Japanese are the world's largest user of robots. 
Counting all classifications of robots the Japanese had approximately 77,000 robots 
in use at the end of 1980 (Yoneraoto, 1981, p. 1). Table 3 presents the Japanese 
robotic use by industry, based on robot costs. It seems that the electrical 
machinery and automobile industries are the largest users of robots. Less than 
1% of IR's are utilized outside of manufacturing. Table 4 presents the 1979 
Japanese usage by production process. It is apparent that over 60% of the usage 
is for moving or positioning a workpiece. This is reflected (see Table 5) in the 
predominance in 1980 robot production of the fixed sequence type of robots. 

Of the many reasons for installing industrial robots, the most prominent 
ones ore to increase productivity and reduce costs. A commonly used measure for 
determining the ecomonics of Installing robots is the payback period. This is 
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Table 1 

Definitions and Classifications of Industrial Robots 


Japanese Industrial Standard JIS BO 1 34- 1 979 


U.S. View 


Manipulitor A device lot handling objecu at detired without touching 
with the handsanj it nat more than two ot the motional capa- 
bilitict tuch at revolution, out*in. up-down, righi-lilt 
travelling, twinging or bending, u that it cun tpaiially Irant- 
ponanobjcct by holding.adhvringto.andtortn. 


INDUS1 RIAL K0B01 S all have arnlikc projccliont and grippvrt thai 
pcrfoim rac;oiy work cutionarily dom. by huinant. The term r utually 
reterved lor machinct with tome form of built-in ccniroltyticm and capable 
of ttand-aionc operation. But in Japan, it alto includvt manipulatorv 
operated by humant. either directly or remotely. 


Robot A robot iv d> lined at a mechanical tyticm which hat llcaiblc 

motion funcliont analogout lo the motion lunc.'ont ol living 
organitr.t or eombinet tuch motion lunctiont with intelligent 
lunctiont. and which jcit in retponse to the hunmn will. In 
thit conical, intelligent funcliont mean the ability in perform 
at Icatl one ol the lollowing: judgment, recugniliuii. t dapla- 
lion or learning. 


Classification by input information 
and teaching method 


Classification by Servo Type 

NON SERVO ROBU1S 

/ A PICR-ANU-Pl.AC'E R0B01 it the timpicti vertion aecouniing loi 
I about oiK-lhird ol all U.S. intlallaliont. 1hc namet comet Item il« irtiial 
I application in malcrialt handling, picking tomething Irom one tjio. and 
tC placing it at another, hrccdom of movement it utually limicd to two or 

It three dirccliont in and out. left and right, and up anJ down I he control 

It tytiemitcicc'romcchanical. Pricct range from S5.(KKMoS.fU.UUtl 


Manual .\ manipulaloi lhai itdireclly operated by a ma i. 
manipulator 


Sc jence A manipulator, the working ticp of which rpcralet tequen- 
r .oot lially in compliance with pretet proceduret. tondiliont and 

potiiiont. 

hixcd A tcqucncc robot tit dclinco above, lor which the prcM.i 
tcqucnce information cannot be catily changed. 


A SERVO ROBOI it the moti .ommon indutiria! robol because it can 
include all robots detrnbed below Ihctwmc ttcmsliomoneor mure sen o- 
mcchanitmt that ciiabic ihc aim and gripper to alter direction in niidaii, 
without having to trip a me _nical twitch. Live to seven directional move- 
ments arc common, depending on the number ol ~joinlt.~ or ailicutal.ons, 
in the roboi't-arm 


Variable 

SCUUCJICC 


Playback 
rt oot 


A tcqucnce robol at dclined above, for which the prctri 
inlormaiion can be easily changed 


A manipulator that can repeat any t'pcraiion alter being 
intirucicd bv a man. 



Numerically A manipulator that can execute Ihc commanded operation 
controllco m compliance with Ihc numerically loaded working informa 
‘ lion on e g. position, tcqucncc and conditions. 



A PROliRAMMAHLE ROBO'I it a servo robol directed hy a program- 
mable conirvillerihul mcmori/ci a tcqucncc of >trm-and-gripper moscmcnis. 
this routine can then be lepeatcd perpetually. 1he robot is rcprog.ani'ned 
hv leading its gripper through Ihc new task. 1 he price range it SJS.UCU .o 
SVO.UOU. 

A COMPU'I ERIZFD R0B01 it a servo model rur hy a computer. Ihc 
computer controller docs not have to be taught by leading Ihc aim-grippei 
through a routine, new instructions can be iransmilled cleeuonically. 1 he 
programming lor sucli -smart* robots may inrludc the ability to opiimire. or 
improve, its work-roulincinsiruclions. Prices start at about kt.t.UOU 


Intelligent 

robot 


A robol that car determine its own actions through its sensing 
and rccogn'live abilities. 



A SENSORY BOStC. isacompulcri/cd robot with one nr more arlilici-'.l 
senses, usually sighi or touch. Prices lor early models start at about S73.ll(l<i 

AN assembly ROBOI is a compulcri/ed robot, probably a sensory 
model, designed ipecifically lor assembly -line jobs. Eoi light, hulcli-menu- 
lacluring applications, the arm's design may be lairly anthropomorphic. 


Souicc' Yonemolo. IVKI. P S 


Source Business Week. June V. IdkO. P M 
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Table 2 


IRs in Operation 
(As of the End of 1979) 


Japan 

lU.OOO Units 

( 69 ^) 


* 

United Spates 

3*255 

(16^0 

— 


’.vest Germany 

850 





Sweden 

600 



Total 





20, 

,260 

Italy 

500 



(100:1) 

Poland 

360 

Western 





Europe 



France 

0 

0 

3,013 





(15%) 



Britain 

185 





Norway 

170 





U.S.S.R. 

25 





Belgium 

13 






(Note) Manual Manipulators and Fixed Sequence 
Robots are not included. 


Source: badamoto, 1981* P» 52. 
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Table 3 


Principle Users of Industrial Robots In Japan 

Share in I98O 
(Based on Value) 

Electric Machinery Industry 36H 

Automobile Industry 30 % 

Plastic Moulding Products Industry lOjt 

Metal Working Industry % 

Metal Working Machinery Industry 
Steelmaking, Textile, Chemical, 

Shipbuilding and Misc. 15% 


Source: Yonemoto, I98I, p. 2 
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Table U 


JAPANESE ROBOT USAGE BY 

PRODUCTION PROCESS 

Process 

% of Total 

Loading & Unloading 

UO 

Transfer & Sorting 

21 

Palletizing 

9 

Welding 

6 

Work Maintenance 

U 

Assembly 

3 

Spraying 

2 

Other (including 
Pouring, Screwing, 
Riveting, etc. ) 

15 

Total 

100 


Source: Research Div. , Long-Term Credit Bank of Japan, 

1981, p. 28. 


5 


Table 5 


JAPANESE PRODUCTION OF ROBOTS BY TYPE IN I98O 


Type 

Unit 

Value 

($ Million) 

Manual Manipulators 

l , 9 h 0 (10%) 

13 (»*%) 

Fixed Sequence 
Control Robots 

13,UU0 (67%) 

lOU (31%) 

Variable Sequence 
Control Robots 

1.3U0 (7%) 

1*3 (12%) 

Playback Robots 

2,030 (10%) 

71 (21%) 

NC Robots 

990 (5%) 

97 (29%) 

Intelligent Robots 

130 (1%) 

1C (3%) 

Sub-Total (100%) 

Other Robot -Related 
Devices 


10 

Total 

19,870 (100%) 

3U8 


Source: Ogawa, 198l» P» 132 








usually given (see f c r example, Simons, 1980, p. 32) as the cost of the robot 
and accessories (including fixtures, etc.) divided by the annual savings: 


P 


I 

L-E 


where p - payback period In years 
I = total Investment 
L <= total annual savings In labor 
E = total annual robot upkeep cost. 

Typically, automation Investments have required that the payback period be three 
years or less, with actual payback periods reaching as low as one year when 
multiple shifts are utilized. 
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II. Primitive Functions Accomplished by a Robot 


Industrial robots are primarily designed for manipulation purposes. The 
primitive actions Imposed at the tiand by the manipulator are: 

1 . Moving 

- from point to point 

- following a desired trajectory 

- following a contoured surface 

2. Changing Orientation 

3. Rotation 

Non-servo robots can provide a sequence of point to point (PTP) motions. 

For each motion, the manipulator members move full tilt until the limits of tres'el 
are reached. Thus they are often referred to as '*bang-bang” , "pick and place" 
or "limited sequence" robots. "Programming is done by setting up the desired 
sequence of moves and by adjusting the end stops for each axis" (Tanner, 1979, 
p. 7). The use of mechanical stops and limit switches yields good positional 
accuracy, which is typically repeatable to better than + 0.5 mm (Simons, 1980, 

p. 20). 

Servo-controlled robots can be designed to provide a sequence of PT^ motions, 
with controlled Intermediate velocities. They can also be used to follow a pro- 
grammed trajectory or by using sensors to follow a contoured surface. 

If we consider end effectors, the robot can also be made to grasp, push 
and pull, twist, use tools, perform insertions and assembly, and do other mani- 
pulations done by humans. 

If sensors and appropriate computations are added, the robot can perceive, 
inspect, recog lize, test and do many of the other perceptual functions now dene 
by humans. 

All applications require that the robot interact with its environment in 
the execution of its programmed task. Thus the robot is often designed to 
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switch machines on and off auu to accept Information from machines as to their 
status. The robot may also have provisions for accepting signals from a moving 
assembly line, so It can track it. Similarly, the robot may have Interface 
provisions for tactile, visual or other sensory Inputs for feedback control. 
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III. Manipulator Arm Configurations 


To accomplish these primitive actions, four basic configurations of 
manipulator arm designs have evolved: 

(a) Rectangular (Figure la) 

(b) Cylindrical (Figure lb) 

(c) Spherical (Figure Ic) 

(d) Anthropomorphic, articulated or 

jointed arra (Figure Id) 

These arras utilize both rotary ("revolute") and linear ("prismatic") joints, as 
appropriate, to accomplish their function. Six degrees of freedora (6 DOF requires 
6 joints) are necessary to locate the end effector (hand) at any point and orient' 
ation in the work volume. More DOF's are sometimes used to prevent interference 
of the arm with the work piece or other obstacles. As few as 2 or 3 DOF may suffice 
for a simple "pick and place" robot. 

As indicated in Figure 1, only three DOF's are required to position the end 
effector. However for applications requiring a high degree of wrist flexibility, 
as many as three of the arms DOF's are located in the wrist. These DOF’s are 
usually referred to (using aeronautical terms) as pitch, yaw and roll. 

The basic four manipulator arm configurations can be combined in various ways 
to produce a bewildering set of manipulator combinations. These combinations can 
be further extended by employing angled arms and offsets. Often built-in 
mechanical linkages are added to do such things as constrain the motion of nominally 
articulated arms to cylindrical coordinates for ease of control. 

In addition, the arms may be designed to be mounted on their sides, hung 
from the ceiling, or mounted on rails for mobility, to further multiply the 
range of installed configurations. 
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Figure 1 

Basic Manipulator Geometries 
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Sphoricai . 

(Polar Coordinatea) Articulalad or Jointod Spharical 

(Ravoluta Coordinataa) 


IV. Actuators 


There are three major ways today of powering the actuators that move 
the manipulatoi joints: pneumatic, hydraulic and electrical. Pneumatic is 

simple and inexpensive but "squooshy" and is best used for "pick and place" 
robots whose trajectory motions are controlled by mechanical stops. Hydraulic 
actuation (currently the most popular system) is capable of high-power for a 
given size, but suffers from leakage problems as well as requiring pumi 3, 
storage tanks and other accompanying paraphenalia. Electriccil-mechanical 
actuation is clean but doesn't offer good power to weight ratios. Stepper 
irotors are inexpensive but sometimes lose pulses and therefore accuracy. 

torque motors are very reliable, offer good controllability, but can be 
expensive. Because the overall cost advantage of hydraulic systems diminishes 
with size, electrical drives are to be preferred for the small robot. 

Actuators are the most problem-prone area of robotics. Thus considerable 
research is needed in this area to further improve reliability (robots have 
an approximately 98% up-time these days) and to reduce costs and weight. 
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V. Mobility 


At present, most robots in industrial applications are fixed Inplace or 
ride along rails, guideways or conveyors. Simple robot carts for transporting 
materials are now being used In industrial and cotmnerical applications. Robot 
vehicles using wheels and treads have been researched by NASA for planetary 
exploration purposes. 

Crawling robots, using their manipulator arms to grasp handholds, are being 
explored by NASA for use in constructing large structures in space. Walking 
machines are still in the experimental stage, with research going on at Ohio State 
University, Carnegie Mellon University (CMU) , Tokyo Institute of Technology, and 
in the Eastern Block countries. Engelberger (198C, p. 63) states that "Practical 
robot devices with legs could have advantages in such applications as arctic trans- 
port, mining, agriculture, forestry, fire-fighting, explosive ordance disposal, 
unmanned ocean-floor surveys, and planetary exploration". 
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VI. End-Effectors 


The hand or gr inner, sometimes called the "end-effector", (attached to the 
robot wrist) can be a mechanical, vacuum or magnetic device for part (or object) 
handling or for tool manipulation. At present, most of the these Interchangable 
devices are grippers which are simple open-and-close devices. The variety of 
tools and grippers that can be adapted for robot use is virtually unlLuited (see 
for example, Engelberger, 1980, pp. 41-58). The end-effectors are usually unique 
to the robot application and thus customarily provided by the user, though several 
manufacturers offer a broad selection of devices for grasping parts that might be 
adaptable to the particular task being performed. Nevertheless, though hundreds 
of different special-purpose end-effectors now exist, the end-effeccor remains 
one of the major limiting factors in universal robot manipulation due to lack of 
dexterity and programmability of the hands. Extensive research and development 
is now underway to produce grippers that can handle a wide variety of part 
configurations. 
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VII. Control 


Control of robot arms ranges from highly repeatable open loop devices 
to servo-coatrollers that utilize external sensors in the control of robot 
actions. 

Open loop devices may be as simple as a step sequencer and positlonable 
mechanical stops for a pick and place robot, to more complex devices using 
stepper motors to reproduce a desired motion. 

Servo-devices can use internal sensors, such as joint position sensors, 
or external information sensors such as force, proximity devices, and even 
vision (operating under computer control) . 

Servos operating only on internal sensors require very careful positioning 
of the work-piece (which can require expensive fixturing or feeding devices) . 

Servos utilizing data from sensing the external environment, require more 
complex processing of the signal, but yield much more flexible systems. 

The controller, in addition to controlling the manipulator motion, often 
also serves as an interface to the outside world - coordinating the robots 
motions with machines and assembly lines and ^urning on and off machines that it 
is operating. 

A. Pick and Place Robots 

The simplest controllers are non-servo (open loop) devices that rely on 
sequencers and mechanical stops to control the manipulator end- point positions 
along each of the axes. These "limited sequence" robots are also referred to 
as "pick and place," "bang-bang," or "fixed stop" robots. These robots have no 
provision for trajectory control between the end points. 

B. Programmable Robots 

Programmable robots are servo controlled robots of two Utioic types; "point- 
to-point" and "continuous path." 
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Point-to-point robots are directed by a programmable controller that memorizes 
a sequence of arm and end-effector positions. Hundreds of points may be memorized. 
The robot moves in a series of steps from one memorized point to another under 
servo control using internal joint sensors for feedback. Because of the servos, 
trajectory control oetween the memorized poxnts is possible and relatively smooth 
motions can be achieved. 

Continuous path robots do not depend on a series of Intermediate points to 
generate a trajectory, but duplicate during the playback process the continuous 
motions recorded during the teaching process. Thus, these robots are used for 
painting, arc welding and other processes requiring smooth continuous motions. 

C. Computerized Robots 

Computer-controlled robots are capable of being programmed "off-line" using a 
high level programming language and do not have to rely on being physically taught. 

D. Sensory Robots 

These are computerized robots that Interface to the outside world thru external 
sensing such as sight or touch. These "intelligent robots" are capable of adapting 
to a variety of conditions by changing their actions based on relating the sensed 
information to their goals or preprogrammed decision points. 

E. Ro*->ots as Part of a Flexible M a nufacturing System 

A flexible manufacturing system (FMS) is a programmable batch-processing 
arrangement that contains programmable machine tools and transfer devices all 
under central computer control. When robots are used in such a system, the robot 
controller may itself be just the bottom layer of a hierarchical control system 
for the FMS (c.f. Topperwein, et al., 1980, pp. 96-105, or Albus, 1981A, pp. 

261-280) . 
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VIII. Sensor-Conr.rolled Robots 


Sensor-controlled robots are the first step on the path to truly 
intelligent robots - robots that can deternine their own actions based on 
their perception and planning abilities. 

The simplest versions of sensor-controlled robots utilize contact 
switching for stopping the arms and for opening and closing grippers. 

More sophisticated robots are now beginning to use couch, force and 
torque sensing. Forces and torques can b? derived from stress measurements 
or from internal signals (e.g. derived from back EMF) . Tactile infoiraation 
may be derived from def lection-indured resistance variations of a special sensing 
surface, or by various other ingenious approaches. 

Work in France and at MIT on "artificial skins" with embedded 
arrays of touch sensors, opens the possibility of a robot eventually being 
able to determine position, orientation and identity of parts by touch 
alone. This may be particularly important for difficult assembly 
operations (Kinnucan, i981) . Proximity and range sensirc '^-e also now 
being considered. 

Vision, the most complex of all the sensory modalities, is now 
beginning to find its way into robc-tic control. 


IX. Robot Vision 


In the Industrial manufacturing arena, to circumvent the problem of requiring 
that the workpiece be In a prescribed position and orientation (pose) for the robot 
to operate upon it, sensory systems are now beginning to be employed. Vision 
provides perhaps the nK>st flexible approach to avoid all the flxturing that 
would be required to achieve a fixed pose. 

Figure 2 lists the desired functions of a machine vision system to achieve 
fully flexible sensor-controlled manipulation. Figure 3 Indicates potential ap- 
plications for such a system. 

A. Methods for determination of pose ; 

1. Range can be determined in four principle ways: 

Stereo* 

Triangulation 

Active ranging (e.g. time of flight of light or sound) 

Using optical focusing. 

2. Orientation can be determined by : 

' Observing the relationship of three (or more) known object 
points that are not all colinear in the viewing field, pro- 
vided that the relative ranges connecting these points are 
known . 

Deriving surface normals using the intrinsic image concept 
(Barrow and Tennenbaum, 1981). 

* Deducing it based upon the response of lighting on the object, 
utilizing known charicteristicF or the object. (This is simplified 
if structured lighting such as sheets of light are employed). 
Interferometry measuring the phase differences of an oscillating 
signal received at multiple points. 
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Figure 2 


Desired Functions of Machine Vision for Sensor-Cont rolled 
Meinlpulatlon In Industrial Manufac Corin g 


* Recognition of vorkpie'^-es/asseoiblles and/or 
recognition of the stable state vhere necessary. 

* Determination of the position acid orientation of 
workpieces /aLSsenib lies relative to a prescribed set 
of coordinate axes. 

* Extraction amd location of salient features of a 
workpiece/aissembly to establish a spatial reference 
for visual servoing. 

" In-process inspection — verification that a process 
has been or is being satisfactorily completed. 


Source: Rosen 
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Figured 

Block Diagram of Potential Vision Applications in Robotics 











B. Feature Extraction 


Though it is possible to operate directly on an analog output of a vIJicon 
imager (by such approaches as optical correlation, or template matching) , it is now 
fashionable to use solid state imaging irrays, such as CCD or CID devices, which 
directly provide points (pixels) In the image. 

These pixels are then digitized by intensity l*».vels (usually 2 to 16) sc 
as to be put in a convenient form for operation upon by a computer. From this 
data, edges and regions are determined. Edges are considered to occur when there 
is an intensity gradient exceeding a selected threshold. These gradients are 
derived from intensity differences between adjacent pixels (often using an averaging 
scheme to reduce noise effects) . Regions are grown by annexing ("clustering") 
adjacent pixels that are close to the same intensity level. Various schemes 
(heuristics) exist to connect lines having gaps, smooth irregularities, and remove 
discontinuities in regions (all resulting from noise in the image). The result 
(which can be considered as a form of data compression) is a reduced image that 
consists of edges and regions. Furthe- data compression might be achieved by re- 
presenting edges by sets of straight or curved lines (the ends and curvature of 
which are all that need to be known) . 

C. Object Recognition 

Having determined edges and regions, it is possible using the methods 
of Section A to determine position and orientation. However, often it is 
necessary to determine which object is being observed from a collection of 
possible objects. This may be accomplished by 

L. Template Matching ; Using a stored image in the computer. 

2. Use of Edge and Region Statistics ; This may be curvature of edges, 
area moments of inertia, ratio of square of perimeter to area, number 
and/or location of holes or notches, etc. 
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3. Fourier Transform ; Determining the coefficients of the spatial 
Fourier transform and comparing them to the coefficients of known 
oi'Jects. 

D. Future Techniques 

In the future we may see: 

Greater use of direct range measurements - e.g. scanning laser 
radar, or LED's and photo diodes using phase or Intensity measure- 
ments. 

Parallel processing to obtain feature extraction In real-time. 

Improved methods for determining edges and regions. 

Use of optical markings for recognition, for optical code reading or for 
determination of pose. 

Use of intrinsic image analysis - to determine surface orientation, 
depth, relationship of foreground to background, and texture. 

Scene analysis for less-structured situations. 
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X. ProgrammlnR a Robot 


There are four primary methods for programming a robot: 

Phys leal-setup , in which the operator sets up programs by physically fixing 
stops, setting switches, arraying wires, etc. This is characteristic of the 
simpler robots. 

Lead-through , in which the operator leads the rooot through the desired 
positions and locations by means of a remote teach box. (These points are 
recorded and used to generate the robot traj^‘Ctory during operation.) 
Walk-through , where the robot arm is physically manipulated through 
the desired motions (which are recorded and then played back by the 
robot-control during operation) . 

Writing a software program , which is then executed when desired. 

The emphasis in programming research today is on software programcixng of 
computer-controlled robots. Work on sensor-controlled manipulation is extending 
the scope for programmability. Interacting v»ith the robot by means of software, 
provides more flexibility then the other programming methods, and allows for 
conditional actions or flexible adaptations. Various high level robot programming 
languages su-'h as VAL (Unimation) and AML (IBM) are now beginning to become available 
to aid in the software generation. 

Simons (1980, p. 107) notes that software programs can be divided into two 
types: "In explicit programming , the user requires explicit instructions for 

every action the robot must t'»ke. In world modellin g, the robot is more 
autonomous and can make decisions according to its knowledge. World-modell ing 
systems, largely in the research stage, tend to require a considerable amount 
of computer power but are able to carry out complex tasks. The most iirportant 
world-modelling languages are AL, AUTOPASS, and LANA..." 
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XI. Klnetaatlcs and Dynamics 


A. Cooidlnate Transformations 

If the arm Is programmed by walking-thru, then It Is simply necessary to record 
the simultaneous joint positions during the walk- thru. If the arm Is led-thru, then 
In more sophisticated systems, that can operate lu tool point, world, or other co- 
ordinate systems. It Is necessary to do the needed coordinate transformations for 
each joint In order to obtain the desired end-effector orientation and position. 

This Is particularly true when the arm Is being programmed by %nrltlng software. 

Vfhen the arm Is controlled via feedback from a vision or other sensor. 

It Is necessary to do coordinate transformations to convert the camera or 
hand coordinates to base or computer coordinates. Fom vrlslon (or other sensor) 
control, the sensor Interpretation and coordinate transformations need to be done 
In real time. As the mathematics for exact transformations requires time-consuming 
calculations, approximate simplifications have been developed to speed the calculations. 
In some systems, Individual microprocessors are used to process the kinematics at 
each joint. 

Note, that arms are often designed so that, the wrist motions are mechanically 
constrained to cylindrical or rectilinear coordinates to simplify control and re- 
duce coordinate transformation requirements. 

B . Trajectory Selection 

For moving the end effector from one position (or point) to another, various 
schemes are possible for choosing the motions for moving the joints. 

Approaches for articulated arms Include: 

Variations from preprogrammed paths. 

The "dominant axis" approach: some joints may be allowed to adapt 
while others follow fixed movements. 
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All joint motions move proportionately relative to the slowest joint, so that 
all joint motions terminate at the same time. 

Considerations of this sort become particularly Important for off-line or real-time 
programming of computer-controlled robots. 

C. Dynamics 

At present, all robot amui are made relatively stiff so that structural 
flexibility Is minimized (to promote repeatability). However, this requires 
stiff and usually heavy arms with high power actuators, lowered carrying capacity, 
and reduced speed. Research Is now underway on the control of structurally 
flexible arms to help overcome these limitations, and In active control to control 
flexure for achieving on-line precision under load as during assembly or drilling 
operations. The 15 m6:ter long Romote Manipulator System (RMS) for the Space Shuttle 
Is probably the most extreme case of a structurally flexible manipulator thus far. 
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XII. Research Requirements 


I A. Different Views 

In considering research requirements It Is very helpful to differentiate 
between near-term needs and far-term needs. Figure 4 Indicates the near-term 
needs for improvement in IR's, as perceived by Japanese users. Hote the 
emphasis is on robot flexibility and versatility of movement, speed, and ease 
of programming. 

In observing how private enterprise Is responding to the IR user desires, 
1 Umentanl (1981, p. 81) indicates that the robot manufacturers most Important 

targets for development Include: 
reduced production costs 
smaller Installation space requirements 
greater flexibility 
larger carrying capacities 

modular robots components to enable a building hlocV approach to 
diversification of functions 

visual and tactile feedback and recognition systems 
Table 6 lists the main research subjects being pursued by Japanese 
Robot Manufacturers. These may be regarded as relatively near-term. 

Note the correspondence with user desires In Lhelr emphasis on speed, 
flexibility and programming versatility. 

As we move towards the longer term, we encounter robot research 
such as is being pursued In Japanese Universities and Japanese Government 
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Figure 4 

1980 JIRA User Survey of Desired 
Improvements In Industrial Robots 
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Sourca: Umantani, 1981, p. 84. 
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Main Reseeurch Subjects In IRs 
1980 JIRA Survey of Japanese Robot Manufacturers 


Order of 
Importance 

SubJ ect 

No. of 

Research 

Subjects 

% of 

Total 

1 

Increased Speed 

49 

13.1 

2 

More Compact Size 

36 

9.7 

3 

Computer Control 

32 

8.6 

k 

Lighter Weight 

30 

8.0 

5 

Interchangeability 
by Module 

25 

6.7 

6 

Object Recognition 

19 

5.0 

7 

Iiicreeised Payload 

17 

4.6 

8 

Improved Actuators 

14 

3.8 

9 

Self Diagnosis 

lU 

3.9 

10 

Adaptive Control 

14 

3.8 


Source: IMentanl, 198l» p* 98* 
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Research Institutes. Here a 1980 JIRA research survey Indicates (see 
Umentanl, 1981, pp. 95-96) that their primary research topics (In decreasing 
order by number of research projects) are: 
control 
locomotion 
vision 

tactile sensing 
manipulators 
end effectors 

programming methods and tools 

Figure 5 lists the desirable goals for future robots (from a U.S. 
vantage point) as seen by Engelberger (1980). He states (p. 12*^) that 
of the missing robotic attributes, the two most crulcal ones are vision 
and Lactlle sensing. As Indicated la Figure 6. this view Is shared by the 
partlcpants In the NBS/RIA Robotics Research Workshop. 

Kelley and Huston (1980) , Birk and Kelley (1981) , and Albus (1981B) 
have also attempted to come up with research requirements for future 
robots, though they did not attempt to prioritize them. Their requirements 
are summarized in Figure 7. In the following discussion we will combine these 
research needs into categories for ease of presentation. 
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Figure 5 


Source: 


Desirable Goals for Futur e Robots 


1. Rudimentary sense of vis.w to provide 

a) recognition data 

b) orientation data 

2. Tactile sensing to provide 

a) recognition data 

b) orientation data 

c) physical interact ion data 

3. Computer interpretation of the visual and tactile data 
U, Multiple appendage hand-to-hand coordination 

3* Computer directed appendage trajectories 
6. Mobility 

7* Minimized spatial Intrusion 

8. Energy conserving musculature 

9. General purpose hands 

10, Man-robot voice communication 

11, Total self diagnostic fault tracing 

12, Inherent safety (Asimov's Lavs o. Robotics) 

13* All the capabilities above available for a price vh5.ch allows 
purchase and operation within the traditionally accepted rules 
for economic Justification of any new equipment 


Engelbarger, I 98 O, p, II 8 , 
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Figure 6 

"Oixaended Distribution of U.S. Robotics Research Funds/ Effort 


% 

Binary Vision lA 

Gray Scale Vision 12 

Off-line Programming 12 

Control Systems 11 

Touch, Force, Torque Sensing 11 
End Effectors 10 

Integration Into CAM Systems 8 

Mechanical Systems 7 

Simulation and Modeling 6 

Safety 4 

Mob ili*_y /Line Tracking 4 

Others 1 

•• ♦ 

100 % 


Sourcer NBS/RIA Robotics Research Workshop, 1979, p. 49. 
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Figure 7 

Basic Elesearch Needs in Robotics (not prioritized) 


Kelley and Huston ^.1980) 

Manipulator Kinematics 
Contr ^ 

Locomotion 

Sensing 

Artificial Intelligence 
Higher-Order Languages 
Smaller Actuators 


Albus (1981B) 

Accuracy 

Speed and Dexterity 

Sensing 

Control 

Knowledge Representation 
rrogramming (Higher Order Languages) 
Standardized Interfaces (Modularity) 
Mobility 


Blrk and Kelley (1981) 

1. Intelligence 

Knowledge Representation 
Auto. Prob. Solving, 

Decision Making, Planning 

2. Control 

Flexible Structures Control 
Modeling 

Real-time Sensory Control 
Fault Tolerance, Monitoring 
and Repair 

3. Actuation 

4. Manipulation and Hands 

Performance Evaluation 
Kinematic Planning for 
Assembly 
Speed 

5. Dexterity 

6. Locomotion 

7. Sensing 

Vision 

Tactile 

Real-time Computation 

8. Programming 

9. Adapatlon and Learning 
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B. Synthesis of Nee ‘ed Research 


1. Manipulator and Actuator Design ; 

To achieve the higher speeds, flexibility, accuracy, efficiency and 
dexterity that is desired in future robots, additional research in manipulator 
arms are needed. For large manipulators and precision work, joint and arm structural 
flexibilities can be a problem. Therefore stiff, but lightweight arms are 
desirable to minimize this problem and to aid in increasing speed. However as 
variable compliance in different directions is often desirable to provide 
the "give" needed for inserting parts and other assembly operations, unique 
design approaches may Lc required. 

At present, actuators are one of the more problem-prone areas of robots. 

In addition, actuator efficiencies are in the order of one tenth that of Human 
musculature. Smaller, more reliable and efficient actuators and drive mechanisms, 
with high load to weight ratios, are needed for the hign speed, lighter weight 
robots of the future. 

For greater dexterity and less intrusion into the work-zone, it is also 
necessary to understand the mutual interdependence of end effector aud mani- 
pulator kinematics. Also research on micro-manipulators is needed for applications 
to micro-electronics and biology. 

A unified theory is needed on how to design an arm for smooth universal 
movements, satisfaction of functional requirements, and to provide the required 
load capacities throughout its motion spectrum. A standardized approach to per- 
formance measurement and evaluation is required to enable comparisen of robots 
not only in such factors as working volume, load capacity and reliability, but 
also in aspects such as speed, smoothness and accuracy throughout their operating 
range. 
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To help determine manipulator accuracy (now limiting lactor in ofi line 
programing of open-loop manipulator operations), a universal stochastii. landel 
of mechanical errors of manipulators needs to be devised. 

2. End Effectors 

Umetani (1981, p. 110) rotates that "the most difficult barrier in industrial 
robot design... is the design of dexterity in hands and fingers." Tliu.f , much 
work is needed to devise desi'^s for general purpose hands (and the neans to 
control them), such that the whole arm need not move to provld > the d'=.slred 
dexterity. Because general purpose >^nds are not yet available, multiple end 
effectors may be required for a single application. This in turn requires some 
type of quick cha.ige wrist design. 

3. Control 

Kelley and Huston (1980) indicate that further research is needed on the 
basic robot problem: given the desired external or absolute displacement, 

velocity and acceleration of the hand, determine the opiimal physical and 
internal motion parameters. (Birk and Kelley, 1981) state that procedures 
have been recently developed for efficiently obtaining and solving governing 
dynamic equations for robotic systems with flexible links and joints. However, 
optimizing the arm motion still presents problems of convergence and uniqueness 
(Kelley and Huston, 1980). 

Though the repeatibility of arm positioning via open-lcop (no external sensory 
feedbu'-k) robot -:ontrol has been good, accuracy for initially positioning the arm 
via open-loop control has been poor. In addition, no efficient means of robot cali- 
bration has yet been determined. Thus it is important that more work be done on 
improving cpen-loop control accuracy. Accuracy degradation due to flexure can be 
a particularly troublesome problem when trying to perform precision operations under 
load, such as drilling without the aid of jigs. 
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An important area of control not yet solved is control of structurally 
flexible manipulators. This problem, important to high speed manipulation with 
manipulators having high load- to weight ratios, may require real-time simultaneous 
management of a large number of command and control variables. An associated 
problem trlth such manipulators may be the large changes in moments of inertia 
as the manipulator is extended and a lead applied. Another problem associated 
with high speed manipulation, requiring additional research, is the generation of 
time-optimal trajectories. 

An area fom which little is known about ideal solutions, is the control of 
redundant-axis manipulators used to provide additional manuever capability. 

Design of control systems for fault tolerance will become particularly Im- 
portant as robots become more autonomous. 

Finally, lack of suitable strategies for control of multi-fingered hands is 
still one of the principle factors retarding design of dexterous hands. 
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A. Sensory-Controlled Mar Ipulatlon 


One of the most desirable features for improving robot flexibility Is 
sensory capability. Vision, the most popular U.S. robot research area, ar^^ears 
to be the most desirable sense for present and future IR's. Next in importance 
is touch. 

To take full advantage of sophisticated sensory data, real-time control using 
sensors is needed. Hierarchical control may be the appropriate means to handle 
such data, though control logic systems such as pattern reference feedback logic may 
be appropriate in certain contexts. 

In addition to vision and tactile sensing, dynamic control using force feedback 
is an important area of research. Also of Increasing Importance as robots become 
more autonomous, is proximity sensing as used before final contact and to avoid 
obstacles. Other sensors will Increasingly have roles in in^'pectlon and In 
determining the type of material being observed. Research is needed in the use of 
models in sensory processing and Interpretation. 

It is also necessary to explore the tradeoff between the use of sensors and 
imposing core structure ou the robot environment. As sensory control is perfected, 
the tradeoff will undoubtedly shift towards the direction of using sensors. 

5. Vision 

Vision becomes more important as the environment becomes less structured. 
Therefore vision is needed more for fire-fighting, household servants, robot 
explorers, etc., than for IR's. Vision can also reduce manipulator accuracy require- 
ments, by making real-time adjustments practical. 

At the current state of the art, additional research in recognition and geometric 
representation is needed. As greater capability is desired, three dimensional vision 
combined with world models and spatial reasoning will Lecome important. Current 
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3-D research in vision systems center on the use of range-finding, structured light, 
and binocular vision systems. Further-out, research is required in the analysis of 
image sequences (optical flow: shape from motion) , intrinsic image analysis (deter- 

mining shape from a single image by disambiguating the response to light) , and 
Improved methods to recognize objects in clutter. The bin-picking problem is a 
classic one in this latter area. 

One of the most Important research areas in vision is approaches to Improve 
the speed of visual processing. This includes research in computational ''laments 
structured for visual processing, which Includes parallel-processing CPU’s, and 
special chips for edge-finding, Fourier analysis, etc. 

6. Tactile and Force Sensing 

What is needed is better use of contact sensing data. This includes better 
resolution in touch and force sensing. An Important result could be the ability 
to recognize parts and to determine their relative position and orientation through 
such methods. 

7 . Programming a Robot 

Programming by teaching is not practical for small lot production, particularly 
when sensory interaction is involved. Thus, Improved techniques are needed for 
generating robot control data. These could Include high-level programming languages, 
software debugging tools, world models, and the ability to directly utilize CAD 
data bases and sensory inputs. Robot simulation tools can aid in off-line programming. 
Also needed are better geometric modeling systems. As robot usage advances, it 
would be desirable to produce aids for writing programs to orchestrate 
multiple arms and multiple sensors, and for systems with distributed control. 

8. Intelligence 

Plantier and Bodmer et al. (1981, p. 63) assert, "Artificial Intelligence is the 
technological area which needs most to be developed and mastered to accelerate 
robot evolution." Birk and Kelley (1981) indicate that an intelligent r«. bot is 
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one capable of : 

receiving communication 

understanding its environment by the use of models 
formulating plans 
executing plans 
monitoring its operation 

Thus research in all these areas is important. 

Communication ; Use of a natural language interface, or computer graphics or 
even voice commands would all be helpful in simplifying communication with a robot. 
In addition, intelligent aids for robot users to help plan and develop robot usage 
are also needed. 

Knowledge Representation ; ^ key ingredient in advanced robotics is knowledge 
representation. However, the tradeoff between control algorithm complexity and 
representation complexity needs to be explored. Sophisticated internal models 
of thto environment are needed to generate expectations, to Interpret sensory data, 
and for us-3 in automatic generation of robot controx data (such as for real-time 
decision-making) . 

Representations are also needed to enable qualitative reasoning about physics 
and space for use in planning and problem solving. 

Parallel processing will be needed to make real-time modeling a reality. 
Planning ; Automatic problem-solving or planning is one of the keys to future 
intelligent robots. Planning is even needed for acquiring information from 
sensors. Needed is research on ways to use large amounts of domain-specific 
knowledge to guide planning and responses to recognized situations. 

One of the difficult, but important areas of research needed is in 
developing plans that deal with time. 
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For certain tasks, the software for handling contlngences Is becoming 
excessive. Thus means for automatic monitoring and correction are needed. 

In addition, plan-monltorlng and repair Is also In need of research. It would 
also be useful to be able to store successful plans In categories so that they 
can be reused. In general. It Is desirable that the system be capable of adaptation 
and learning. 

Automatic assembly In non-hlghly structured environments Is a key research 
area. Development of kinematic strategies for assembly Is needed. Assembly could 
be aided by model-based generation of programs that use local sensory information 
In assembly tasks. Research Is also needed for planning and orchestrating the 
actions of cooperating robot systems for assembly and other purposes. 

Intelligent controls can be thought of as coupling control theoretic approaches 
with advanced AI methodolgles. To advance this area, better understanding of modular 
hierarchical structures is needed as well as how to generate hierarchical plans 
associated with such structures. 

To make intelligent robots really useful, it is necessary to develop methods 
that will enable them to function in real-time (not having to pause f^r computation) 
with realistic computers. 

9. Robot Mobility 

For industrial robots tending machines, furnaces, etc., the robots are often 
in use only a frac".lon of the time, the remainder being in a wait mode. To make 
better use of the robots, machines are frequently arranged so that one robot 
can terd several maihlnes. Sometimes this involves moving the robot (motion along 
tracks being the usual means). Locomotion is needed to further extend the working 
range and for use in hazardous environments and in remote locations. In many cases, 
a truly oiobile robot would be preferred that can be used where needed. Mobile robots 
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would be particularly advantageous in environments less structured than that found 
In factories, such as In the construction of ships, buildings and roads, 1*> 
mining, and In undersea or space use. 

New inechanlcal mechanisms, other than wheels, are needed for mobility over 
uneven terrain. However, mobility systems tend to amplify compliance and 
vibration problems (which can be minimized by caveful design and the use of sensor- 
driven control). Active suspensions are needed for uneven terrains, climbing, etc. 

For legged vehicles, the key problems are stability, control and optimal motion 
synthesis. In walking machines very little Is known about the stability of various 
gaits. Advances are needed in the hierarchical control of multi-body systems. 
Efficient algorithms need to be developed for solving the governing dynamic equations. 

If the mobile robot is to be supervisory-controlled, then optimal partitioning 
of the task between the operator and machine becomes an important research arc-a. 

10. Hodularlty 

Further wcrk Is needed not only to develop modular manipulators, but also 
to enable Integrated modular systems that can consist of a variety of robots, tools, 
sensors and computers all connected together. To do this, standardized interfaces are 
needed. It will be necessary to partition the control problem Into modular 
components and to develop Interface standards for communication between these 
components. An important goal is to be able to write an off-line program 
for one robot and be able to use it with a differently designed robot. 

The 1980 NBS/Air Force ICAM Workshop on Robot Interfaces (1981) considered the 
following items for standardization. 

* simple sensor interface between simple peripheral devices and a robot 
control system. 
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Wrist interface between the robot wrist and the end effector. 


Complex sensor interface that covers vision, complex touch 
and other such sensors. 

Common robot control Interface, providing robot Independent 
trajectory descriptions. 

Robot Prograamiing Languages 

Integration of robots into robot systems for automated manufacturing. 

It was concluded that both the simple sensor and wrist Interfaces were 
ready for standardization, but that it was too earl}' in the development of 
the other areas to begin a standards effort at that time. 
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XIII. Conmerclally Available Robots and Components 

Engelberger (1980, p. 117) Indicates that many of the attributes required 
to make Industrial robots '^tactical In an Industrial environment are available 
today. These (see Figure 8) Include good positioning repeatability, reliability, 
flexibility, easy programmability and adequate load capacity, all at a affordable 
price. 

The rapid growth of Industrial robots (roughly 35Z per year) and the entry 
of new manufacturers and new products makes Its inappropriate to try to list or 
ev%n suimnarlze all the robots currently available. However, the following 
are good sources of Information to keep current with available robot products: 

Robotics Age , a bimonthly magazine popularizing robots, P.O. Box 725, La Canada, 
CA 91011. 

Robotics Industry Directory published yearly by Robotics Age , P.O. Box 725, 

La Canda, CA 91011. This directory lists cotiplete Information on over 
one hundred different models of currently available robots, robot components 
and peripherals. It also provides Information on robot dlstrlb.-cors, 
r bot research Institutes and consultants in. the robot field. Robot Systems 
and Products , providing a monthly new products update. Is also available from 
Robotics Age . 

Specifications and Applications of Industrial Robots In Japan published by 
Japan Industrial Robot Association (JIRA) , 3-5-8, Shlba Koen, Mlnato-ku, 
Tokyo, Japan, A 1982 edition is currently available. JIRA is a good source 
for all information on Japanese robots. 

Robotics Today , a quarterly magazine on industrial robotics published by the 

Society of Manufacturing Engineers in cooperation with the Robot Institute 

0 

of America, One SME Drive, P.O. Box 930, Dearborn, MI 48128. 
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Figure 8 

Robot Attributes Nov Commercially Available 


1. Work space command with six infinitely controllable 
articulations between the robot base and its hand 
extremity 

2. Teach and playback facilities - realizing fast, 
instinctive programming 

3. Local and library memories of any practical size desired 

• 4 . Random program selection possible by external stimuli 

5 . Positioning accuracy repeatable to within 0.3 mm 

6, Weight handling capability up to I 50 kilos 

7» Point-to-point control and continuous path control, 
possibly intermixed in one robot 

8. Synchronization with moving workpieces 

9 . Interface allowing compatibility with a computer 

10. Palletizing and depalletizdng capability 

11. High reliability — with not less than uOO hours MTBF 

?..2. All the capabilities available for a price which allows 
purchase and operation within the traditionally accepted 
ruled for economic Justification of any new equipment 

Source: Engelberger, 19^0, p. 117. 


^3 



The Industrial Robot , a quarterly magazine with a European orientation toward 


Industrial robots, published by IFS Ltd., 35-39 High St., Bedford MK42 7BT, 
England . 

An excellent recent guide tc Industrial robots for perspective .'ers - 
Sanderson, R. J. et al. , \dustrlal Robots; A Summary and Forecast for Man u- 
facturing Managers , 1982 - can be purchased from TECH TRAN Corp., 134 N. Washington 
St., Naperville, IL 60540. 

In addition, various robotic newsletters (such as "Robot News International," 
also available from IFS Ltd., "Industrial Robots International," available from 
Technical Insights, Inc., P.O. Box 1304, Ft. Lee, N.J. 07024; and the "Robotics 
Newsletter," available from Bache, 100 Gold St., New York, NY 10038) offering 
information on new robot pioducts and investments have recently begun to appear. 
Datamat jcu , By*~e , Business Week , Time and other technical and popular magazines 
have also recently begun to have feature articles on robotics. 

At the end of 1981, there were some 50 U.S. based firms marketing or pre- 
paring to market robots (Conigllaro, 1981), the leaders being: 

‘ Unlmatlon Inc. (projected to have 32% of the estimate $215 million 
(3075 units) U.S. market In 1982) and Cincinnati Mllacron (projected to 
have 25% of the U.S. market In 1982). 

Several of the large U.S. auto, electrical and computer firms have been building 
robots for internal use and are now entering or considering entering the com- 
merical robot market. 
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Japan at the end of 1981 had over 130 different manufacturers producing 
robots, many for their own internal use. Table 7 presents the robot production 
by major Japanese manufacturers for the years 1980 and 1981. Of the estimates by 
JIRA of 19,400 robots, valued at 77 billion yen, produced in Japan in 1980, only 3% were 
exported. 

In 1979, approximately 2400 robots were sold in Europe. European manufacturers 
having a major share of this market include: 

ASEA - Sweden 
Olivetti - Italy 

The Eastern Block Communist countries have been moving in the direction 
of robots to make up for their serious labor shortages. Business Week (Aug. 17, 

1981) reported that the Soviets plan to manufacture 40,000 robots by 1985 to 
counte_ not only the shortage of skilled labor, but also absenteeism associated 
with drinking. 
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Table 7 


Number of Robots Produced by Major Japanese Robot Manufacturers 


Note: The figures In brackets Indicate approximate value. 



1980 

1981 

Major Product 

Company 

Achievement 

Target 

Function 

Kawasaki Heavy 

450 

650 

Spot welding 

Industries 

($22 M) 

($31 M) 

Arc-welding 

Yaskawa Electric 

360 

630 

Arc-welding 


($16 M) 

($29 M) 


Hitachi 

150 

480 

Spot welding 



($22 M) 

Arc-ueidlng 

Mitsubishi Heavy 

120 

240 

Spot welding 

Industries 



Prlnt-spraylng 

Kobe Steel 

120 

200 

Arc -welding 


($7 M) 

($13 M) 

Paint-spraying 

Star Seiki 



Plastic mould 



($13 M) 

picking 

Shlnmelwa 

360 

480 


Industry 

($4 M) 

($9 M) 

Arc-welding 

Fuji Electric 

130 

200 

Inspection 



($9 M) 


Fu jitsu-Fanuc 

480 

600 

Machine processing 

- 

($4 M) 

($7 M) 

Assembling 

Sankyo Seiki 

_ 

300 

Assembly 



($4 M) 


Nitto Seiko 


120 

Assembly 



($3 M) 



Source: August 1981 Survey (Survey Japan, 1981, pp. 210~211). 
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XIV . Commercially Available Vision Systems for Robots 


Present commercially available visiou systems for use with robots are 
basically devicer that generate and analyze 2D images. These systems recognize 
objects by their silhouettes. Principle current manufacturers of sophisticated 
computer vision systems are: 

Machine Intelligence Corporation (M.I.C.) 

Sunnyvale , CA 

Automai'.ix Inc. 
durlin^’.ton, MA 

Octek Inc. 

Burlington, MA 
M.I.C. Vision Module 

This Vision Kodule is based upon prototype software and hardware developed 
at SRI under NSF and industrial affiliate support (see Gleason and Agin, 1979). 

The Vision Module consists of a solid-state TV camera, a preprocessor inter- 
face and a LSI-11 microcomputer. The vision software, stored in the LSI-11 memory, 
includes the entire library of vision subroutines previously developed at SRI, 
several application programs (such as training-by-showing and execution of part 
recognition), as well as newly developed capabilities. The capability to develop 
application programs by calling the vision-librai^ r’jbroutines makes the Vision 
Module general-purpose vision system. The Vision Module is Intended for a 
variety of industrial vision applications such as inprocess inspection, sensor- 
controlled manipulation, and visual servoing. 

The Vision Module can recognize parts on the basis of their size and shape 
regardless of their position or orientation. The part (lying in one of its 

stable states previously used to train the module) is recognized on the basis of 

a set of shape descriptions, including area, perimeter, number of holes, 
length of major and minor bounding ellipse axes, first and second moments of 
area, etc., and various algebraic combinations of these. 

The Vision Module operates by appropriately thresholding the image in the 
preprocessor to achieve binary silhouettes of the objects in the scene. A frame 

buffer stores the binary image (consisting of zeros and ones) to achieve timing 
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Independence betvreen the camera and the concrolling ricrocomputer. Each line 
is sequentially scanned ai.d "run-lengch" encoded (i.e., edge points recorded 
where the pixels change from rero to one or vice versa). Each run-length segment 
on a line is matched against segments of the previous line to determine their 
overlapping relations hips. Using there relation ships, the program traces the 
appearance and di.-appearance of blobs (regions) as the image is processed from 
top to bottom. Only one line at a time of image data needs to be stored in the 
computer. 

At the heart of the vision processing are connectivity analysis routines 
which segment the image into contiguous regions (called "blobs") e>^en in the 
presence of noise. During the connectivity analysis, sequences of perimeter 
points are extracted. Using this blob boundary information, other desired 
parameters such as area, area moments, shape, position and orientation of the 
blob can be computed. 

The system is trained by showing the object to the TV camera resulting in 
all potentially useful shape descriptions being automatically calculated and 
stored. Then the same object is moved and photographed several times and the 
mean and variance of these descriptors calculated and stored. This resulting 
set of means and variances, can be used as a "prototype" for that object or 
class of objects, hew objects can than be recognized in either of two ways. 

One, by using a normalized nearest-neighbor method, where the best fit of the 
features of the new object to the set of prototypes is selected. Two, using 
a binary decision tree (which can be built by the Vision Module software upon 
command) recognition is done by measuring one feature at a time, and at each 
stage subdividing the recognition problem into two smaller problems until a 
single choice remains. The time to read in, process and Implement recognition 
is in the order of one second. 
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Complete vision systems cost from $30-A0 thousand dollars. M.I.C. also 
markets the VS-110 Machine Vision System version for real time, inspection tasks. 

M.I.C. which is currently rapidly expanding, is placing increased emphasis 
on tur.ikey systems, and improved man-machine interfaces. The problems of resolution 
speed and cost are also being vigorously attacked. 

The Automatix Autovision System 

The Autovision System (Rheinhold and Vanderbrug, I960) also uses t^*® SRI 
algorithms as its main base for vision processing and in many ways is similar 
to the MIC system. However, the Autovision System's preprocessor (Scheinman, 

1981) has a pixel memory capable of storing several Images at once, or several 
bitr; of grey level Inforjiatlon, or any combination. The versatility of the 
system coiaes from the system's powerful AI-32 control processor, which with its 
32 bit internal design and megabyte memory address space, can operate in parallel 
with the preprocessor and has the ability to interface with multiple preprocessors 
and various other input and output devices. The control processor operates under 
an Automatix developed operating system. Most of the system software is factory 
programmed in PASCAL, except where speed or compactness is so critical that assembly 
language is used. 

The primary interface for the user is via the Automatix application language, 
RAIL, which is designed for the non-computer oriented user. This powerful and 
flexible robot programming language makes it possible to sequence the operations 
of the Autovision system so that the desired task can be performed. 

Octek Vision System 

Octek has several vision systems based on a circuit board developed in 
1980. It can handle two high resolution solid state TV cameras with 320 x A80 
pixel arrays. The system has an on-board microprocessor that can manipulate 
images at four fram j per second. The system can handle a 16 gray level scale. 
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There are four memory planes so that similarities and differences between images 
can be determined. The system relies on a Data General cr DEC computer for 
computation. The basic circuit board costs about $4500, with the cost of a 
complete vision system ranging from $16,000 to $40,000. 

KEYSIGHT AND CONSIGHT 

At General Motors, several different computer vision systems are now 
operational (Rossol, 1981). KEYSIGHT is an example of a systeir for inspection 
tasks in automobile manufacturing. It uses a lo%f-resolutlon solid state camera 
to inspect for the presence or absence of valve spring cap keys on automobile 
engines. Ingeniously applying fairly standard computer vision algorithms 
for edge finding, etc., the system locates the center of the valve spring as- 
sembly and then, based on the keys being brighter than the holes they fit over, 
determines if the keys are present. 

More interesting is CONSIGHT, an example of a robot vision system that 
uses structured light to detect the silhouette of passing objects. Using a robot 
arm, CONSIGHT transfers a moving stream of non-touching, randomly oriented parts 
from a moving conveyor oelt to a predete'Tuined location. 

Structured light provides a means to deal with situations that require 
both high spatial resolution and fast response. It allows part geometry to be 
sensed directly, as opposed to being inferred through grey level picture pro- 
cessing. 

CONSIGHT is logically partloned into Independent vision, robot and monitor 
subsystems. When the vision subsystem has seen the entire object, it sends to 
the monitor the object's position and belt position reference value. The robot 
subsystem then executes a previously "taught" robot program to transfer the 
part from the belt to a fixed position. 


A single narrow and Intense line froBi two parallel, separated sources 
Is projec.ed across the belt surface perpendicular to the belt's direction of 
motion. / linear array camera builds a conventional two dimensional silhouette 
Imate of passing parts by collecting a sequence of these image strips. For each 
object detticted, a small number of numerical descrlptirns is automatically 
extracted lor part classification and position determination. 

CONSICHT is now coismercially available, having been licenced to Unlmatlon, 
Cincl>nati llilacron, M.I.C. and Automatix. 
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XV. current Robot Research 


In Japa n 

Robot related research In Japan can be placed into three categories - resi;arch 
done by universities and research Institutions, by Industry, and by spe.rlal govern- 
ment sponsored taeV forces. A 198C JIRA survey found that a total of 88 research 
projects were then underway fil universities and public and private research 
institutions. However unlike the U.S., there were no large-scale research efforts 
at the uni*/ersities or technology colleges. 

Areas of emphasis in research projects at Japanese universities and research 
institutions are (Hasega\'a, 1981, pp. 162-163): 
increasing robot speed 
coordination of multiple arms 

dexterous robot hands with tactile recognition capability for 
automatic assembly 

sensory development for intelligent robots 
robot locomotion 

robot control and application software development 
In the United States 

Robot research in the U.S. is primarily aimed in the direction of making 
robots more Intelligent and versatile, with vision research being given a high 
priority. Albus (1981B, p. 7) states, "Much remains to be done in sensor technology 
to improve the performance, reliability and cost effectiveness of all types of 
sensory transducers. Even more remains to be done in improving the speed 
and sophistication of sensory processing algorithms and special purpose hard- 
ware for recognizing features and analyzing patterns in space and time." 

Robotic research efforts at U.S. universities (roughly reflecting FY-82 
funding) are indicated in Table 8. Note that the research is centered on greeter 
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T.ible 8A 


Robotics Research 
Institution: Stanford University 

Principal 

Researchers: Thomas 0. Blnford, Bernard Roth, 

Robert Cannon, Larry Lelfer 
(23 Grad Students, A PostDocs) 

Research Funding: Approx. $1,400 K/year from NSF, ARPA 

and Ni^A 


Manipulation 

* Control of flexible manipulators 

* Control of multiple manipulators 

* Design of three-fingered hand 
Perception 

’ Robot Vision 

* Stereo Mapping 

* Special purpose VLSI for vision 

■ Segmentation 

■ Force sensing and control 
Programming and Intelligence 

■ AL (arm language) one of the most advanced robot programming 

languages available today. 

* ACRONYM system for geometric modeling and reasoning for model- 

based vision and for programming robots. 

* Work on CAD, resulting In a simulator for mechanical systems 

and robot arms, and for off-line programming. 

Mobility 

* A continuing program of navigation and perc< ntion. 
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Table 8B-1 


Institution ; Artificial Intelligence Lab., M.I.T. 
Research Funding: $1.75M DARPA and ONR. 


I. Robotics Mechanisms and Control 


Principal Researchers: Hollerback, Seerlng, Mason, Pubrlck, Salisbury, 

Asada, Lorenzo Perez, Brooks, Hlllls, 


Manipulation 

Design of Dexterous Hands 

» Tendon-actuated three-fingered hfuids (for use with PUMA) lighter, 
nimbler, more compliant. 

- Controllers for tendon-actuated fingers and shoulders. 

Manipulator Design 

- Direct drive actuation 

- Developed a recursive Lagraglan formulation of the dynamics of 
r^anlpulators. 

- Developed a simulation for robot linkages and biological arms. 

- Hybrid controller that allows the available degrees of freedom to be 
divided Into position servclng and force feedback servolng as neeced 

to achieve the compliance required for assembly and operation under load. 

Perception 

Force and tactile sensors for automating compliant assembly task. 

- High resolution tactile sensor. 

- Strain gage force sensor. 

Prograsmlng and Intelligence 

Developing a high level language based on a geometric modeling system 
developed at the AI Lab. 

- Allows errors to be mod«>led, as well as keeping track of spatial relation- 
ships in ways programmers find natural. 
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- Connecting a PUMA directly to u LISP machine to be able to program It 
In this geometric modeling language - to Improve robot speed » accuracy 
and Interfacing with sensors and effectors. 

IT . Robot Vision for Parts Assembly and Repair 

Principal Researchers: Brady, Crimson, Horn, Larson, Nlshlhara, and Pogglo. 

Perception 


Acquire possibly overlapping laminar workpieces off a textured background. 

- Real time edge detection carried out In hardware nslng a camera developed 
at M.I.T. 

- M.I.T. crnstructed frame grabber provides a Interface to commercially 
available area array cameras for real time processing of Image files. 

Utilize 3D vision for guiding the acquisition of objects from a bln of 
parts. 

- Photometric stereo to determine local surface orientation. 

(1) Using multiple light sources and assuming Lambertian reflectance. 

(2) Use of diffuse light sources to determine layout of sceuces containing 
specular objects. 

- Binocular stereo vision utilizing matching of detected edges to obtain 
depth. 
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Table 8B-2 


Institution: Mechanical Engineering Dept., M.I.T. 

Principal Researchers: Thomas B. Sheridan, Nam F. Suh, Nathaniel H. Cook, 

Neville Hogan, (10 grad, students) 

Research Funding: Approximately $500K/Yr. from NOAA, ONR, rlSF and NIHR. 

I. Use of Robotics (Teleoperators) Underseas 
Manipulation 

Teleoperators under computer control. 

Perception 

Tactile sensing for oceanbottom operations. 

Measurement arm attached to base of robot to subtract out base motion. 

Vision tradeoffs of video frame rate, resolution and number of gray 
scale levels. 

II. Upper Extremity Prosthetics 

- movement control 

- adaptive control stiffness 
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Table f,C 


INSTITUTION: Robotic^ Institute, Carnegie-Mellon University 

DIRECTOR: Raj Reddy 

RESEARCHERS: 15 Faculty 

6 Research Associates 
18 Graduate Students 
23 Staff 

RESEARCH FUNDING: Approx. $4 M/year from Westinghouse> Digital 
Equipment Corp, ONR, DARPA, NSF, and other 
industrial sponsors. 


Manipulation 

- Direct drive rare-earth magnet electric actuators for manipulator joints 

- Multi-processor control systems 

- Dexterous end effectors for handling compliant parts 
Perception 

- Laser range sensor 

- Proximity sensors 

- Binary and gray scale Image analysis and understanding 

- Touch, force, pressure, vibration, temperature, and chemical sensors 

- 3D sensors 

Mobility 

- Mobile cart 

- Gait and legged motion analysis 
Manufacturing 

- Factory of the Future Project 

- integrated factory system 

- research testbed for manufacturing problem 

- Intelligent Factory Management System 

- Job-shop planning and scheduling 

- Factory modelling and analysis 

- Flexible user interfaces 

- Flexible machining 

- multiple robot and machine integration and control 

- Flexible Assembly 

- multiple robot and sensor integration and control 

- multipl* processor controllers 

- electronic assembly 

Programming and Intelligence 

- Knowledge representation 

- Problem-solving techniques for planning and scheduling 
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Table 8D 


Institution: Rhode Island University 

Principal 

Researchers: John Blrk 

Robert Kelley 

Research Funding: Approx. $200 K/yr. from NS? 

Approx. $1 M/yr. from Industrial Associates 

Manipulation 

* Dexterous end effectors 

* Articulated hands (includes manipulation within the hand itself). 

* Design methods for hands 
Perceptio n 

* Visual Pose estimation 

* Touch sensing 
Programming and Intelligence 

■ Robot prcgramming language 
Manufacturing 

* Successfully demonstrated bin-picking 

* General methods to enable robots with vision to acquire, orient and 

transport workpieces 

* Design of r'roductlon systems utilizing robots 
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Table 8E 


Institution; University of Florida 
Principal 

Researcher: Del Tesar (-f-10 faculty from 5 disciplines 

and 20 grad, students) 


Research 

Funding: Approx. $1 M/yr. from DOE, NSF and State of 

Florida 


Manipulation 

• Human augmentation with real-time digital computer assistance 

(computer-based teleoperators) 

• Robot geometry, kinematics and dynamics - to achieve high-level 

dexterity 

• Control of structurally flexible manipulators 

• On-line robc.t precision under load (for batch processing) 

• Actuator modules - to enable construction of customized manipulators 
Programming and Intelligence 

• Hierarchical control in an unstructured environment - using force feedback 

assistance. 
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Table 8F 


Robot Research at Other Universities 


University 

$ 

Yearly 

Pv’nding 

Manipulation, 

Perception, 

Mobility 

Programming 
and Intelligence 

Manufacturing 

Purdue 

lOOK 

NSF 

• Robot Vision 

• Robot Control System 

• Robot Progranmiing 

Languages 

• Modeling of 

parts flow thru 
in industrial 
plants 

U. of Mass 

300K 

NSF 



• Design of parts 
for automatic 
assembly 





‘ Economic applic- 
etion of assembly 
robots 

U. of Mich 

500K 
State 
of MI 

• To Implement a 
new Robot Lab 



U. of MD 


• Robot Vision 



U. of 
Rocbeoter 

200K 

NSF,0NR 

• Machine Vision 

• Computer graphics 
language - PADL 

• Automated 
Meuiufac tu ri ng 

U. of 
Arizona 

lOOK 

• Teleoperators 



Ohio State 
( Robert 
McGhee ) 

NSF 

& DARPA 

• Legged locomotion 

• Manipulator Dynamics 
and Controls 


U. of IL 
U. of PA 
U. of WA 
U. of Texas 


• Small projects in 
robotics and 
related work 




R.P.I. 
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dexterity, improved sensory perception, highe’"-order nbot programming languages, 
increased robot intelligence, and computerized manufacturing and assembly. As 
indicated in Table 9, the focus o' robotics research in the U.S. non-profit re- 
search laboratories is on manufacturing, «rith emphasis on computerized planning, 
factory management, assembly and inspection. 

Table 10 indicates robotic research efforts in commercial U.S. companies. 

The emphasis is on manufacturing, assembly and computer vision for part recognition 
and inspection. 

Information on research and development in some of the U.S. companies special- 
izing in sophisticated computer vision is Indicated in Table 11. The emphasis is on 
vision for inspection and robot control, and on simplified command capability. 

Table 12 gives an indication of the research activities of the principle U.S. 
robot manufacturers. Emnhasis is on advanced control systems, improved pro- 
gramming techniques and mechanical design. 

Thougn there is no central focus for robotxcs in the U.S. Government, several 
government agencies are involved in robotics research and a number of others are 
looking into app'lications. 

Table 13 (roughly reflecting FY-82 funding) provides an overview of robotics 
research performed within the U.S. Government. In general, these activities can 
be vieved as exploratory research into robotic applications. 

Table 14 presents an overview of robotics research (roughly reflecting FY-82 
funding) supported by the U.S. Government. NSF primarily supports basic research, 
ONR supports a mixture of basic and applied research, while the V.A. is focussed 
on potential applications for the handicapped. 

Table 15 (again roughly reflecting FY-82 funding) indicates some of tlie govern- 
ment supported robotics applications development in the manufacturing and rework 
a reas . 
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The various U.S. military services are considering future robotic research 
efforts that could lead to replacing humans, cr leveraging their capabilities, 
for dangerous Jobs or missions and for increased productivity. 

It is estimated that the current (FY-82) overall U.S. Government support 
for robotics research and development is approximately 18 million dollars per year. 
Research in Europe and the Soviet Union 

The Western Europeons are estimated to be spending two to four times as 
much as the U.S. on robotics and related research. (.V recent survey of teleoperator 
and robotics research in Europe indicates that $10 million per year is being spent 
in telemanipulation R&D alone, (Plantier and Bodmer et al. , 1981, p. 8).) 

As in the U.S., the empnasis is on making robots more intelligent and versatile. 
"Germany, Italy, Sweden, France, Switzerland and the United Kingdom, either in 
national research laboratories or in robot manufacturers, are presently the most 
advanced European countries in computer vision and sensor-based robots" (Plantier 
and Bodmer et al., 1981, p. 12). 

No figures are available on robotic research in the U.S.S.R. and the Eastern 
Block Countries, but they appear to be making a strong effort in robotics in order 
to improve productivity and overcome labor shortages. However, the Soviets lack 
of ready access to digital and computer circuitry is hindering their research effects 
in advanced robots (Teschler, 1981, p. 48). 
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Table 9A 


Robotics Research 

Non-Profit Lab: Charles Stark Draper Lab, Inc. 

Principle 

Researchers: Jim Nevlns 

Dan Vfhltney 

Research Funding: Approx. $1 M/yr. from ySF and Industry 


Manufacturing 

■ Part-mating science for assembly operations 

- Sensory feedback 

- Adaptive learning 

- Passive cumpllance 

-Models of assembly process 

« Economic and configuration modeling for making choices 
among competing technologies In designing Industrial 
systems 

• Flexible assembly system design 

• Operational strategies 

• Product design for produclblllty 

Applications, with industry, range from precision products with 
’ow annual volume to products with annual volumes in excess of several 
million. Techniques developed are also applicable to other environments, 
such as space, underwater, and medical. 
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Table 9B 


Robotics Research 


Non-Profit Lab: 


SRI International 


Principal 

Researcher: David Nltzan 

Research Funding: Approx. $1 M/yr. from NSF and 

Industrial Affiliates 


Perception 


• Machine vision for Inspection and recognition 

* Robot vision Involving structured light, a combination of 

binary and gray-scale vision, and local features to 
distinguish overlapping parts 

Manufacturing 

• Vision-guided arc-welding 

* Printed circuit board inspection 
■ Programmable assembly 


6A 


Table 10 


Company 

GM 

Warren, MI 


GE 

Schenectady, Nif 


Westinghouse 
Pittsburg, PA 


IBM 


Texas 

Instruments 

Martin Marrletta 
Denver, CO 
($3M/yr.) 


Robot Research in Commercial U.S. Companies 


Research Activities 


Robot vision systems for inspection and part transfer 
(KEYSIGHT, CONSIGHT) 

* Automatic small parts assembly 

* Flexible computer-controlled high-speed precise assembly, 

to handle different versions of a one product fanily from 
a data base 

* Evaluation of existing robots for use in GE manufacturing 

* Robotics Research Laboratory evaluating use of commercial 

robots 

* Supports productivity research at CMU ($lM/yr.) 

* Adaptable Programmable Assembly System (APAS) ($lM/yr), 

cost-shared with NSF 

* Supports Research on Teleoperators for Nuclear Power Plants 

at U. of Florida 


• Developed own robots for its manufacturing operations 

• Developed robot piTOgrammlng languages - AUTOPASS and 

EMILY 

• Developed own robot for assembly and testing of hand 

calculators 

• Robotics research directed at DOD and NASA interests 

- Autonomous devices 

- Automated diagnosis/checkout 

- Speed requirements of space manipulators 

- Simplified high-speed coordinate transformations 

- CAD robot program planning system 

• Two arm coordinated manipulation (for DOE) 
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Table 11 


Near-Term 


Automatlx 
Burlington y MA 


Machine Intelligence 
Corp. 

Sunnyvale, CA 


R&D In Sophisticated Computer Vision Systems 

* Robot Vision 

* Robot Microcomputer Control Systems 
■ Robot Programming Language (RAIL) 

* Commercial Robot Appllcatlr'ns - arc welding 


* Computer Vision Systems for Turnkey Inspection 
■ Materials Handling and Assembly Systems 

* Unlvlslon System developed In cooperation 

with Unlmatlon for the PUMA robot - programmed 
In VAL robot language. This Is the first 
coimnerlcally-avallabla seeing robot. 

* Research in person/robot communication for 

non-computer oriented people. 
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Table 12 


Research Activities of U.S. Robot Manufacturers 


Unimation 

* Advanced control systems 

■ Calibration techniques 

* Programming techniques 

* Mobility systems 

* UNIKA" vision for arc welding 
Cincinnati Milacron 

* New control system architectures 

■ Programming language 

* Mechanical design 

Other Robot Manufacturers 

* Similar activities - all aggressively developing new and 

Improved product lines . 
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Table 13 


Or^anlaatlon 

National Eareau 
of Standards 
$1-1/2 M/yr. 


In-House Government Robotic Research 


Research Activities 


Standards 

• Robot Interface Standands 

• Robot Performance Measures 

• Programming Language Standards for 

- robot systems 

- Integrated coopu’:er-alded manufacturing 
Advanced Concepts for 

■ sensory-lntaractlve control systems 

• modular distributed systems 

• sensor Interfaces to the control systems 

of robots and machine tools 

■ vision systems 
Focussed Application 

• Automated Manufacturing Research Facility 

- planning 

- scheduling 

- routing 

- inspection 

- demonstration of a completely automated 

machine shop consisting of computer 
controlled NC machines served by 
robots and robot carts, producing 
mixed configurations of parts on 
demand 
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Table 13 (cont.) 


In-House Government Robotics Research 


Organization 

Center 

Activity 

NASA 

Langley Research Center 

• Robotics Laboratory to 

$lM/yr. 

Hampton, VA 

develop teleoperated 

12 man years 


automated spacecraft 



servicing techniques 


Trajectory planning and 
multi-arm coordination 

- Man-machine interface 
for remote systems 

- Operating system for 
concurrent processes 

In a distributed computer 
network 

- Robotics simi'lation 


Marshall Space Flight Center • Research on end-effectors 
Huntsville, Ala. 

• Man-machine interface for 
teleoperation 

JPL • Research in Hand-Eye 

Pasadina, CA Coordination 

• Control ot structurally- 

flexlble manipulators 

’ Teleoperators 

• 3 fingered hand with 
Stanford U. 

Naval Ocean Systems 
Center 

San Diego , CA 
$65 OK 


Exploring application of robot and teleoperator systems 

* submerslbles 

* land vehicles 

* underwater manipulators 

* stereo optic and acoustic vision 

* remote presence 

* autonomous robots - knowledge representation 

and automatic decision-making 

* complex robot systems specification and 

verification 
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Table 13 (cont.) 


In-House Government Robotics Research 


Exploring the use of telepresence 
(teleoperation with sensory feedback 
directly to the operator, giving a sense 
of being at the work site) for use In 
materials handling aboard ship* 


Naval Ocean Systems Center 

Hawaii 

$300R 


Table 14 


U.S, Government Supported Kobotlc Research 

Organization: NSF 

Washington, D.C. 


Program (FY-81) 

Organizations Supported and P.I. 

Activity 

Production Research 
$2M 

U. of R.I, 

- Blrk 

Part acquisition using 
vision 

(W. Spurgeon) 

C« K* GorpN 

- Hayes 

Part Recognition 

(Emphasis on robotics 

SRI 

- Barnard 

Auto. Visual PCB Inspection 

^or assembly) 

SRI 

- Nltzan 

M.I. applied to Industrial 
Auto. 


Merrick Corp. 

- Cook 

Adaptive arc welding 


Taylor Sys. Engr. 

- Kerth 

Adaptive arc welding 


Stanford U. 

- McCarthy 

& Blnford 

Integrated Programmable 
Auto. 


M.I. Corp. 

- Roseu 

Natural Language 


U. of Mass. 

- Boothroyd 

Economic applications 

Mechanical Systems 
(A. Strauss) 

U.C. Berkeley 

- Tomlzuka 

Digital Control of 
Industrial Process 

$700K 


- Dornfleld 

Control of arc welding 

(Emphasis on 
control, dynamics. 

UCLA 

- Dubowsky 

Dynamics of high speed 
machines 

klnemaf Ics, and 

U of FL 

- Sandor 

Kinematic analysis 

design) 

II 

- Hole 

Nonlinear dynamic structual 
synthesis 


U. of Nebraska 

- Schade 

Mechanism motions 


Rut ge rs 

- Lee 

Design of high speed 
mechanisms 


Stevens 

-• Trlcano 

Optimum design of 
mechanisms. 


Columbia 

- Freudensteln 

Forces In high speed 
linkages 


Ohio State 

- Srlnlvasan 

Control of machine tool 
feeds 


U. of Conn. 

- Huston 

Multi-body dynamics 


Pr o gram (FY-81) Organizations Supported and P»I. Activity 


Automation, Bio- 
engineering and 

U. of PA 

- Rozlna 

Scene understanding 

Sensing Systems 

Purdue 

- Kashyap 

Image modeling 

$750K 

CMU 

- Sanderson 

Parallel signal processing 

(Emphasis on 

SUNY 

- Shapiro 

Edge detection In images 

robotics vision 

U. of S. Carolina 

- Cannon 

Auto signal processing 

and Intelligent 

U. of IL 

- Ahuja 

Dot pattern processing 

controls) 

Purdue 

- Funltunaga 

Image feature evaluation 

(N. Caplan) 

U. of Minn. 

- Wechsler 

Texture analysis 

U. of IL 

- Cose 

Auto, scene analysis 


RPI 

- Sarldls 

Intelligent controls 


Yale 

- Narendra 

Self-learning systems 
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Table 14 (cent.) 


U.S. Government Supported Robotic Research 


Organlralion; NSF 

Washington, D.C. 

Program (FY-81) • Organizations Supported and P.I. 

Public Technology Westlnghouse - Abraham 

(A. Schwarzkopf) 

$450K 


Computer Engineering U. cf Rochester 

(B, Chern) 

$300K CMU 


- Voelcker 

& Requlcha 

- Chang 


Intelligent Systems V?I 
(Y. T. Chien) 

$100K V. of PA 


- Shapiro 

& Harallck 

- Ba jezy 


Activity 

Programmable assembly 

Geometric modeling 

Robot memory 

Geometric reasoning 

Scene understanding 
using active sensing 
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Table 14 


U.S. Government Supported Robotic Research (cont.) 


Organization 


Office of Naval Research 
Arlington, VA 
$1.7M/yr. 

(Marvin Denlcoi:£) 


Defense Advanced Research 
Projects Agency 
Arlington, VA 
(Robert Kahn) 

§3C0K/Yr. 

Veterans Admlnl8t'‘etlon 
Washington, D.C. 

(Dr. Margaret Gianni) 
;$400K/yr. 


Organization Supported 

M.I.T. 

$700K/yr. 


CMU 

$700K 


SRI 

$23UK 

U. of 

Rocliester 

$10UK 

M.I.T. 

$300K 


Applied Physics Lab 
John Hopkins Unlv. 
$100K 


Activities 

• Fundamental 

research In 

- vision 

- manipulation 

- reasoning 

• Autonomous 

Unmanned Underwater 
Vehicles 

« Communication and 
negotiation among 
distributed robots 

• Machine Vision 


* Fundamental research 
In robotic vision, 
manipulation and 
reasoning (In cooperation 
with ONR) 

■ Robotic work station 
for quadriplegics 


Rehabilitation Engr. Center • Robotic dds for severely 
Stanford Unlv. disabled Individuals 

$200K 

• Super vlsory voice and 
head-orientation 
conunand of a PUMA 
robot 


• Huraan/Machine lnterf<. •• 


VA Hospital - West Roxbury • Myographic wheelchair 
$5UK control 


VA Hospital - White River 
Junction, N.H. 

$65K 


* Communication control 
aid 
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Table 14 (cont.) 


Organization 

Program 

Organizations Supported & P.I< 

Activity 

AaF •OsSvR* 

(\.F. Off. of 
Scientific 
Research) 
Bolling AFB, 

D.C. 

Manufacturing 
Science 
(Tom Waxsh) 

$2 M/yr. 

Developing Educational 
Research centers of 
excellence 

• Assembly In 
manufacturing 


Table 15 


U.S. Government Robotic Applications Development 


Organization Activity 

Naval Air Rework Facilit'’ • Development of robots to 


San Diego, CA 
$3 M/yr. 


Air Force Materials Lab. 

Wright Patterson AFB 
Dayton, Ohio 
$lM/yr, 

U.S. Arn^ Human Engr. Lab. 

Aberdeen Proving Grounds, MD 

Office of Manufacturing Technology 
DARCOM 


- remove rivets and fasteners from 

airplane wings 

- strip and repaint aircraft 

- perform wing assembly 

* Robotic applications in manufacturing 
(Part of ICAM) 


• Formulating a plan for military 

applications of robotics 

* Robot applications In manufacturing 
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XVI. Applications Now and in the Future 


Figure 9 indicates current and anticipated future use of robots in manufacture 
in Japan. Currently, machine tool processing is dominant;, followed by welding, 
assembly, and plastic molding. In Z of value, assembly, and measurement and 
inspection are expected to increase while spot welding, plastic molding and 
machining processes are expected to decrease. 

Table 16 indicates current and 1990 estimates of major uses for industrial 
robots in the U.S. In general the same trends are visible in the U.S. as in Japan, 
with welding and materials handling being currently dominant, moving towards 
assembly being the dominant application in 1990. 

A recent JIRA forecast of robot applications dev- .? opment , introduction and 
diffusion is shown in Figure 10. Observe that by 1983 it is anticipated that robots 
for assembly, inspection, reactor maintenance, cleaning and ocean development, as 
well as al] p^’ases of manufacturing, will be in commercial use, with diffusion 
relatively complete by 1990. In the last half of this decade it is forecast that 
construction and mining robots, robots for fire fighting and e>ren robots to assemble 
soft goods like textiles will begin commercial diffusion. In the same time period, 
prosthetic arms and legs, and robots for use in patient care and other medical 
applications will become widespread. 

In the late 1980’ s, robots for use in various military applications, such 
as use in dangerous missions and hostile environments are also expected to appear. 

Also in the late 80's, robots for use in servicing of space satellites are anticipated. 
In tht 1990' s, robots for use in the construction and assembly of large space 
structures and in space manufacturing are forecast. Starting around 1995, semi- 
autonomous robots are foreseen for use on extra terrestrial surfaces for exploration, 
mining, materials processing, manufacture, and the construction, tending and 
maintaining of space installations. 
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Figure 9 

JIRA Forecast of Japanese User Needs 
for IR’s by Production Process \ 



Sourea: RtMarch DW. of »ha Long-Torn* Crodit Bonk of Japon, 1961, p. 74. 



Table 16 


Estimated U.S. Robot Order Trends By Usage 


4 

Through 1981 

1990 

Materials handling. 
Including machine 
loading and un- 
loadlrg 

25-30Z 

30-35Z 

Assembly 

lOZ 

35-40Z 

Spot-Welding 

35-45Z 

3-5Z 

Arc-Welding 

5-8Z 

15-20Z 

Paint spraying 

8-12Z 

5Z 

Other 

8-lOZ 

7-lOZ 


Source: (Conigliaro, 1981, p, 6) 
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Figure 10 

Roboit Development, 
Introduction and Diffusion 
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Source: JIRA (Hasegawa, 1981, p.171) 
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The general trend is toward robots with ever Increasing intelligence and sensory 
capabilities. Engelberger (1980, p. 135) suggests that robots will eventually be 
entering the service industries, with such diverse applications as garbage col- 
lection and fast food preparation and dispensing. He even suggests that "...a 
[limited] household robot may be practicable before the end of the 1980's decade." 
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XVII. The Expanding Robot Population 


Table 2 presents JIRA's (The Long-Term Credit Bank of Japan, 1981, p. 52) 
estimate of the world's robot population as of the end of 1979. The figures 
shown are for high-quality robots and exclude manually manipulated robots (tele- 
operators) and fixed sequence robots. Note that Japan with 14,000 robots has 
nearly 70% of the world's total, compared with 16% (representing a little 
over 3,000 units) for the U.S. 

Figure 11 projects the U.S. robotics industry growth through 1990, when it 
is estim-'ited that the U.S. will have roughly 100,000 IR's in operation, or 
some 30 times that of 1979. This estimate is predicated on major diversified 
companies - such as GE, Westinghouse, GM and IBM - entering the market as now 
appears to be happening. 

Table 17 presents JIRA's forecast for the production of IR's in Japan through 
1990, while Figure 12 indicates how this production will be distributed. 

Integrating the results from Tables 2 and 17 and Figure 11, Figure 13 presents 
the composite forecast of the industrial robot population through 1990. It appears 
that the world's robot population will pass 1,000,000 in the early 1990 's, and will 
therefore have a major impact on the world's productivity. 

A recent article in Business Week (Dec. 14, 1981) indicates that the robot 
pace is continuing to accelerate and even the figures quoted here may be very 


conservative. 


Mliliont $ 



Year 

Figure 11 

U.S. Robotics Industry Growth 

1980-1990 

Yaar 1980 1981 1982 1983 1984 1985 1988 1987 1988 1989 1990 

Eatimetad Salat 

($ Mllliona) $ 100 150 215 280 395 540 725 875 1125 1520 2070 

Eatinatad Units 

f)/'- 1450 2100 3075 4000 5600 7715 10350 13000 17000 23000 31350 

(Ai;*umad Avg. Robot Cost Approx. $70K Each) 

Compound Growth of Ravanua Par Yaar. 35% 

Source: Cong Haro , 1981 
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Table IT 


Forecast of Aggregate Demand in Manufacturing for All Classes of 

IBs in Japan 


(Unit $ Billion) 




1980 1985 1990 

Annual Growth Rate 

1985/1980 1990/1985 



Domestic Demand 
Exports 

Total 

0.3 0.8 to 1.1 1.6 to 2.2 

Negli- 
gible 0.2 0.1* 

0.3 1.1 to 1.3 2.0 to 2.T 

30$ \U$ 

15^t 

ll*1t 


Annual 
Production 
(Thousands of 
Units ' 

19.7 1*3 to 55 75 to 9^+ 



(Note) The export ratio is based on the JIRA target value of 20$ in 1990. 


Source: The Long-Term Credit Bank of Japan, I98I, P» 69. 
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Figure 12 

JIRA Estimate of Composition of iR 
Production in Japan 
(Based on Vaiue) 



i: RMMVch Mv^ TIm Long-T«nn Credit Bank of Japan. 1961, p. 73. 





XVIII. Future Directions 


Hasegawa (1981, pp. 169-170) makes the forecast of unfolding robot technology 
given in Figure 14. Based on these technological trends, he states that 
"...industrial robots will become smarter, smaller, quicker, lighter, stronger, 
more ingenious, easier to operate, more Intelligent and less expensive that they 
are now." 

Engelberger (1980, p. 118) lists in Figure 5 the qualities sought In future 
robots. He indicates that these qualities will be, for the most part available 
during the current decade. Of the missing robotic attributes, he considers the 
two crucial ones to be vision and tactile sensing. 

JIRA (The Long-Term Credit Bank of Japan, 1981, p. 59) indicates chat the 
technology for robot miniaturization, m- dularlzation, high speed operation, and 
computer control has advanced to the point where it is now available and wll] grow 
in popularity and acceptance in the first half of this decade. Technology for part 
recognition and position determination is also currently available, but won't 
receive widespread popularity until the second half of this decade. Technology 
for color and behavior recognition, automated assembly, and machine tool and 
casting robotics won't become adequately advanced until later in the first half 
of the decade and therefore won't achieve widespread popularity until the secon'^ 
half or the 1980 's. 

One of the greatest advantages of robotics and automation occurs in the 
cases where their use greatly Increases equipment utilization. Albus (1981B) 
indicates that a robot arc welder cannot weld faster than a human, but by keeping 
its torch on the work about 90% of the time, versus no more than 30% of the time 
for a human, it can turn out 3 times as much work in the same time. If tVirougl* 
automation mote shifts can be worked, productivity is even further Increased. 
Unfortunately, present day robots cannot set up their ow'n work, so this reduces 
their productivity advantage. Albus. conjectures: 
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Figure 14 

Forecast of Robot Technology 


* Materials 


Lighter and stronger materials for robot structures will be developed. 
These will Include both organic and non-organic substances. As a 
consequence, the size and weight of robots will decrease. 

* Power Sources 


Small, high-capacity power sources such as nuclear powered generators or 
fuel cells will allow robots greater freedom of location and movement. 

° Actuators 


Small, high performance actuators will appear. Robot designs will be 
more stylish. 

® Sensors 


Small, reliable, high-performance, low-cost artificial eyes, ears, and 
tactile sensors will be developed and even low-cost robots will have 
recognition capability. 

® Control Systems 

Advances in microcomputer technology will dramatically improve robot 
control. Control system costs will also be greatly reduced. 

* Communications Systems 

Remote control of industrial robots will be enhanced through development 
in fiber optics, wireless communcations, etc. 

“ P rocessing Methods 

New, non-mechanical processing methods such as laser beam cutting, 
welding and other high-energy processing methods will reduce the processing 
load on robots and make the design work for them simpler. 

“ Robot Body Structure 


Utilization of biomechanical concepts will result in superior robot 
structures. 

® Software 


Robot use will become easier through the development of robot languages, 
self-diagnosing systems and robot application technologies. Software 
will contribute to reducing the design work needed on robots. 
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(continued on next page) 


Figure 14 (cont'd) 


* Integrated Systems 

Techniques for Integrating production systems composed of men, machines 
and robots as well as their peripheral devices will be found. Further, 
their design proiesses will be computerized. 


Source: Hasegawa, 1981, pp. 169-170. 
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Eventually, welding robots will be sufficiently sophisticated 
to work from plans stored In computer memory and to correct errors 
which may occur during a job. Welding robots will then be able 
to work nights and weekends (four shifts per week) completely without 
human supervision. At that point productivity Improvements over 
present methods of many hundreds of percent become possible. Un- 
fortunately, we are a long way from that today. There are many 
difficult research and development problems that must be solved 
first. Unless the level of effort In software development Is 
Increased many fold, these Improvements will not be realized for 
many years. 

In the metal cutting Industry, Albus (pp. 16-18) predicts. 

During the 1980's, robot sensory and control capabilities 
will Improve to the point where robots can find and load unorlented 
parts, or In some cases, even pick parts out of a bln filled with 
randomly oriented parts lying on top of each other. This may im- 
prove productivity by hundreds of percent because it will make it 
possible to Install robots In many existing plants without major re- 
engineering of production methods.... 

By 1990 robots may begin to have a significant Impact on 
mechanical assembly. There has been a great deal of research effort 
spent on robot assembly. Unfortunately, the results have not been 
spectacular — yet. On the one hand, robots cannot compete with 
classical so-called "hard automation" In assembly of mass produced 
parts. General purpose machines like robots are still too slow and 
too expensive to be economical for mass production assembly tasks. 

On the other hand, xobots cannot yet compete with human assembly 
workers In small lot assembly. Humans are Incredibly adaptable, 
dexterous, as well as fast, skilled, and relatively cheap compared 
to robots. . . 

Nevertheless, progress Is being made and will continue. 

Robot capabilities will gradually Increase. Senrory systems will 
become more sophisticated and less expensive. The cost of computer 
hardware Is dropping rapidly and steadily with no sign of bottoming 
out. Software costs are like.ly to be the major Impediment to robot 
development of the foreseeable future, but even these are slowly 
yielding to the techniques of structured programrilng and high level 
languages. 

Eventually, extremely fast, accurate, dexterous robots will 
be programmed using design graphics data bases which describe the 
shape of the parts to be made and the configuration of the as- 
semblies to be constructed. Eventually, robots will be able to 
respond to a wide variety of sensory cues, to learn by experience 
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and to acquire skills by self optimization. Such skills can then 
be transferred to other robots so that learning can be propagated 
rapidly throughout the robot labor force. 

Sometimes, perhaps around the turn of the century, robot 
technology will develop to the degree necessary to produce the 
totally automated factory. In such factories robots will perform 
most. If not all, of the operations that now require human skills. 

There will be totally automatic Inventory and tool management, 
automatic machining, assembly, finishing, and Inspection systcims. 

Automatic factories will even be able to reproduce themselves. 

That Is, automatic factories will make the components for other 
automatic factories. 

Once this occurs, productivity Improvements will propagate 
from generation to generation. Each generation of machines will 
produce machines less expensive and more sophisticated than 
themselves. This will bring about an exponential decline In 
the cost of robots and automatic factories which may equal the 
cost/perforroance record of the computer Industry... 

Eventually, products produced In automatic factories may 
cost only slightly more than the raw materials and energy from 
which they are made. 

In summary, what we see emerging are robots with Increasing Intelligence, 
sensory capability, and dexterity. Initially, we will see an increasing use 
of off-line programming of computer-controlled robots, using Improved robot command 
languages. Prevision will be made to Include the role of sensors, such as 
vision and touch. In this programming. Later, self -planning will emerge as 
higher and more general commands are given to the robot. At this point, the 
marriage of robotics and artificial Intelligence will be virtually complete. 

At the same time as all this is taking place, robotic hands with Improved dexterity 
(so that it will not be necessary to move the entire arm for fine motions), 
and advanced control systems to support this dexterity, will emerge. Also emerging 
will be robots with coordinated multiple arms and eventually even legs, supported 
by even more sophisticated control systems. 
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As this evolution progresses, information and intelligence will become 
the dominant factor in robotics, with the manipulator devices and sensors 
shrinking in importance to the skeleton that undergirds this dominating 
"ghost in the machine". 
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